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MATHEMATICAL BIOPHYSICS OF SOME NEURAL NETS 


J. B. ROBERTS 
THE UNIVERSITY OF CHICAGO 


Methods are considered for constructing neural nets of the McCul- 
loch-Pitts type giving arbitrary time delays between stimuli and their 
responses. By introducing cycles of neurons, the problem is treated 
from the standpoint of economizing on the number of neurons required. 
A generalization is effected that will realize any unique temporal re- 
sponse pattern from a unique stimulus. 


Time Delay Circuits. The “microscopic picture” of neural nets 
(McCulloch and Pitts, 1943) considers individual neurons and syn- 
apses. Time is regarded as “quantized” rather than continuous, and 
only spatial and not temporal addition of impulses at synapses is pro- 
vided for. 

The construction of a neural net such that any desired finite time 
delay, of N units between the action of a stimulus S and the action of 
a response R, is incorporated in the net, can be made very simply by 
inserting a chain of N—1 synapses between the neuron stimulated and 
that responding. Clearly, we can make the delay as long as we desire. 

However, we wish to devise a net that will give a time delay of, 
say, N time units using less than N synapses. Such an economy can 
be realized in neural nets involving cycles of neurons, i.e., sets of neu- 
rons b,,---, b, connected in “series” such that b, is connected to b,. 
Thus a cycle is a repeating system. 

By using such cycles we can devise, with relatively few neurons, 
nets with time delays of any finite length. In the example shown in 
Figure 1, we have two cycles, of five synapses and of seven synapses 
respectively. The response RF will occur at 36 time units after S. The 
net represented in Figure 1 gives this time delay of 36 with the use of 
only 13 synapses. The dotted lines in the Figure represent connec- 


FIGURE 1 
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tions of other neurons to response R,, which occurs at 19 time units 
after S. In general, this net will give a response for any time delay 
from 2 to 36. Any such response for this net will be repeated at inter- 
vals of 35 time units. Note that the period P of the net is equal to the 
product of the numbers of synapses in the individual cycles and that 
the “range” of time delays this circuit can give is from 2 toP + 1. 

We will now discuss the specific construction of a net for some 
particular time delay O;. We will represent the number of cycles by 
ry and the number of synapses in each cycle by Ni(t=1,---, 7). 

In the construction of the nets under consideration we will use 
the following two notions: 

(1) The effective factors (EF’s) of a net (defined for nets of 
the type shown in Figure 1 only). For these nets there is one EF for 
each cycle, and the EF corresponding to any particular cycle has a 
value equal to the number of synapses in that cycle. 

(2) The highest common normal factor of a set of 7 numbers 
(HCNF,). This is defined by the following relation: 


IN, 
HCNF, =—_———__ (1) 
LCM (N,,---, N,) 
where LCM is the “lowest common multiple.” 
It can be verified that 
LOM (Nz » +++; N,) = P(N, ,-*:>N-e), (2) 


where P is the period of the net. For a circuit similar to that of Fig- 
ure 1 we know that the range is from 2 to P + 1 and therefore also 
from 2 to LCM (N,,---,N,) +1. So, given an O; value, we can state 
the following. If 


LCM (N,,---,N,) +120,7, (3) 


then the net with cycles N,,---, N, will realize a time delay of O; time 
units. 


Combining equations (1) and (3) we get 


IN: 
a1 
—— 
HCNF, 
It will now be shown that the HCNF, of N,,--:, N, is 
FF, 1,°*:, N, is equal to 
the HCF (highest common factor) of the r products 


On 1, (4) 
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Breaking N,,--- , N, into prime power factors, we get 


N,= a. 8. e%... 
Nz = a®. bb. cy... (5) 
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Call dmax , Bmax} Ymax, ‘+: , the maximum a;, $;, y;,---, present in the 
array (5). Then 


( JIN 

41 

HCF Se == Q>4i-Imax 5 b>=8i-Bmax ‘: C*Yi-Ymax Brats 2 

N 2 


j 
Also from (1) and (5) we have 


HCNF, = Q4i-Amax . b=8i-Bmax . C=7i-Ymax aes 
r (7) 
ILN, 


Sy GMINA hoe Naa 


which was to be shown. We will use the above result for the comput- 
ing of the LCM , which in many cases can be done more readily by 
breaking the numbers down into their prime factors and choosing the 
proper ones for the LCM . 
_ From relation (4) we see that there are an indefinite number of 
sets of EF’s for any finite O;. This suggests that one can find a “‘best 
set” (or set of best sets) of EF’s in the sense of the one giving the 
required range and having the least sum. This problem is not solved 
here. It is true that a best set for a given range will have the prop- 
erty (Ni, N;) =1; (¢ # 9), but this condition is not sufficient. If 
N,,-:-, N, are relatively prime in pairs, then we have HCNF,= 1. 
If only one pair, say, (Ni, N;) =d, then the HCNF, > 1. (In this 
case, HCNF’,, = d.) 

As an example find the range of the set of EF’s: N,:=2,N.—4, 
N,=9,N.z=6. We have 

HCNF (2, 4, 9,,6) 

2-4-9-6 _ 2-4-9. 67 


~ HCF(2-4-9,2-4-6,2-9-6,4-9-6) | 12 
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From (4) we have O; S$ 37. The range is therefore from 2 to 37. In 
this case the threshold of the last synapse before the response R would 
be 4 (synapse q in Figure 1). In general, the thresholds of such syn- 
apses (q-synapses) will be equal to the ae es of Coe eee 

ea eats 


~+1 ) , we wish 
HCNF, 


to know which synapses in the individual cycles should be connected 
to a single q-synapse in order to give a particular time delay. To do 
this, we set up the following ordering. Construct a table with r col- 
umns and with LCM (N,,---, N,) rows. In the first column of this 
table write downward the numbers 1, 2,---, Ni, repeating until col- 
umn is filled. Do the same with the numbers 1, 2,--- , Nz in column 
two, etc. Table I is an example of the case for three EF’s having the 


Given a circuit with a range of 2 to ( 


TABLE I 

d, d, d, 
2 1 1 1 
3 2 2 2 
4 1 3 3 
5 2 1 4 
6 1 2 5 
ii 2 3 6 


values N, = 2, N,=3,N;=6. We label the columns d,, ---, d, and 
the rows from 2 to LCM + 1. The last row of Table I consists of 
N,, Nz, N;. The last row of any table constructed in this manner 
will consist of the entries N,,---, N,. The number of repetitions of 
N,,--- , N; in column k of the table is equal to LCM/N;,.- Each row 
number corresponds to one O, value. Thus the range of the row num- 
bers corresponds to the range of the circuit that the table represents. 
Therefore, for every Or value in the range, there corresponds a set 
d,,-+++,d,. By the construction of our table, we see that the sets 
(d,,+++,ds)r, T =2,---, 7, indicate the synapse numbers in cycles 
1, ---, 3 which realize the O; corresponding to each set. It follows im- 
mediately that the sets (d,,--- , d-)r in any such table of r cycles are 
all distinct. 


We will call the set d, ,--- , d, for a particular O, the “description 
number” (DN) for that O7. ; 
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| 2 3 4 5 6 
FIGURE 2 

In Figure 2 we have the circuit corresponding to Table I. From 
the Table we can choose a row number (equal to a desired O7 between 
2 and 7), say 6, and find its set of d’s to bed, = 1, d2=2,d,=5. 
Thus d; indicates which synapse in cycle 7 is connected to q for some 
particular O;. 

When the N;’s are not relatively prime in pairs, there will always 
be certain sets (d,,---, d,) that will not appear in the table of DN’s 
for the circuit. These DN’s not appearing correspond to sets of neu- 
rons that will never fire together. 

The number of synapses used for a particular O,; is equal to 
(N, + N. + --- + N, + 1) while the range of delays is from 2 to 
LCM +1. 

Generalization of the Construction. Examining Table I, we see a 
certain regularity. For example, if we look at the row labeled T we 
note 

T= d,.+' lmod Nj, 
T= -d--+ 1.mod Na, 
Tsds+1 mod N,. 


For any table constructed in the manner of Table I, we have the fol- 
lowing set of relations holding (where ;d; stands for the entry in the 
T-th row and the 7-th column): 


(T) [2to LCM + 1] (t) [r] [T =7rd; + 1 (mod N;)]. (8) 


This congruence (8) enables us to construct a circuit giving a desired 
delay of, say, Or by simply solving the proper congruences. 

The following method of constructing a circuit gives a unique 
temporal response pattern for the action of Rk. If the response F is 
to occur at the times 7,, --- , 7», then we can construct a circuit giving 
responses R,, R2, --- , R, with the time delays of Or, =7—1,:--, 
Or, = 7, — 1 (where S acts at r= 0), and then connect each of these 


responses with the response F that is to occur once for each of these. 
Where we have a sequence of impulses all causing the same re- 
sponse FR , then we have one extra delay introduced in the circuit over 
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FIGURE 3 


that for the responses R; (see Figure 3). This extra delay is intro- 
duced by the synapse labeled Q. With Or still indicating the time de- 
lay between the stimulus S and the final response, we see that we can 
use all the equations given previously if we correct for the extra time 
unit. This can be done by using in them the value of O,’ is given by 


O,’ = Or —1. (9) 
The range for a sequence is from 3 to LCM (N,,---,N,) +2. 
From congruence (8) we have Table II, where the Or. are the 


delays of action of R. With Table II we can find DN’s for any se- 
quence of time delays, provided a suitable set of E'F’s is chosen. Any 
set of HF’s found for the largest time delay will suffice. 


TABLE II 
ae 1) 

07, =0 (mod N;) N,—1 

O07, =1 (mod N;) 1a, 

Or, =2 (mod N ;) 1 

07, =3 (mod N;) 2 

07, =4 (mod N;) > 3 


Or, =N,—1 (mod Nj) || Ni—2 


Or, =N, —1 (mod N;) 


Or, =N,—1 (mod Nj) 


Ge ee ee ae eee wns ow en nn ee nee ee ne 


Op, =N,—1 (mod N;) 
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As an example, find a circuit giving R at times giving delays of 
Or =4, Or, =6, Or, =9, Or, = 18, after action of S. The largest 
O, is Or, = 13. From (9) we have Or = 13-—1= 12, so that if 


N,-N> 3-4 
N,=3,N.—4, then from (4) we have = = Or | —-1= 


HONE oie 
12. Using Table II we obtain: 


Oy, =0 (mod Nj) 2 3 


Op, =1 (mod Nj) 3 4 


Or, =2 (mod Nj) 1 1 


Or, =8 (mod Nj) 2 


With this we can obtain next: 


80, '=9 (mod 8) 2 
Se O7 = 83 (mod 4) 2 
5=07,'=2 (mod 8) 1 
5=0,,/=1 (mod 4) 4 
80, =2 (mod 3) nk 


8=0,7,'=0 (mod 4) 8 


12= 0,’ =0 (mod 3) 2 


2 Or,’ =0 (mod 4) 8 


This final table gives the DN’s of 2,2; 1,4; 1,3; 2,3 for Or, = 8, 
oO; = 5, Or, = 8, Or, as 


The circuit is given in Figure 4. This can be checked by the meth- 
od of activity matrices (Landahl and Runge, 1946). 
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FIGURE 4 


Now if we wish to write Table II more compactly, at the same 
time allowing us to insert n—2 synapses between S and the cycles, 
we can do so with Table III. The y;’s and q;’s in Table III are defined 


TABLE III 

xy d, 

0 N,—a@a+1 

1 N;—q+2 

2 N,;—@a+8 
n—1 N; 

n if 
n+1 2 


by the following congruences, where the O,’s are the time delays and 
the N;’s, the EF’s. (In any row of the table, if Ni —q + p > Ni, then 
it is reduced modulo N;.) We have 


n= qi (mod Ni), (10) 


Or = x3 (mod Ni). (11) 


The range for this case is from n + 1 to LCM + n. Instead of con- 
dition (4), the following holds: 


LCM (N,,---,N,) 2 Or’ Opa>), (12) 


ax min 
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Since only the spread of the time delays, so to speak, determines the 
EF’s. 

We now consider as an example the circuit given in Figure 4 
and make the proper connections to give R at time delays after S 
of Or, — 3; Or, = 6, Or, — 8, Or, =9. We see that Ne 33 Ne = 4, 
N, = 6. With these and the O, values, we get from congruences (10) 
and (11) and Table II the following table: 


From this we see the DN’s are 1,1,1; 1,4,4; 3,2,6; 1,8,1. The circuit 
is similar to that of Figure 4. Again the circuit can be checked using 
the method of H. D. Landahl and R. Runge mentioned previously. 

For another example, find a circuit for RF to act after a delay of 
Or, =11, Or, =14, Or, = 16, Or, = 17, withn=10. From congru- 
ence (10) we have 


10=1(mod3), q=1; 

10=2 (mod4), q=2; 

10=4(mod6), q:=4. 
Also, we have from congruence (11) the x; values for Table III. And 
from relation (12) we see that in this example the EF’s, N, = 38, 


ae —3)12 = On = On) 16. 


N.= 4, N; = 6, will suffice, since 
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10 = 6. This gives us the table: 


Or, ne 5 Xs d, d, d. 
10 1 2 4 1 if 1 
13 1 iL i i! 4 4 
15 0 3 3 3 2 6 
16 1 0 4 1 3 1 


Therefore the DN’s needed are 1,1,1; 1,4,4; 3,2,6; 1,3,1. These are the 
same as in the previous example, as they should be, since we have 
merely inserted 8 time units between S and the cycles, while keeping 
the differences of the Or values constant. We would not have obtained 
the same had we changed the differences of the O7’s. 

The author wishes to express his thanks to Dr. A. Rapoport for a 
critical reading of this paper. 

This work was aided by a grant from the Dr. Wallace C. and 
Clara A. Abbott Memorial Fund of the University of Chicago. 
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By considering the general energy balance between the energy of 
muscular contraction in the extremities, kinetic energy imparted to the 
animal and the energy lost in the lifting of the center of gravity and in 
the possible muscular viscosity, an expression is derived for the speed of 
an animal in terms of the constants which determine its shape and struc- 
ture. The relation discussed previously, namely, that the speed is pro- 
portional to the square root of the length of the extremities, turns out 
to be only a first rough approximation in a rather special case. Reason 


are given for the number of joints actually observed in animal extremi- 
ties. 


In a previous paper (Rashevsky, 1944b) we discussed the dy- 
namics of locomotion of “quadrupeds.” It is better to refer more spe- 
cifically to “quadruped mammals” since, technically, such animals as 
lizards are also quadrupeds though the dynamics of their locomotion 
are very different. On the other hand, the same considerations as dis- 
cussed in loc. cit. may apply to the leaping of a frog or even of a grass- 
hopper. 

In that paper it was shown that velocities of locomotion usually 
observed in most mammals could not be obtained with rigid extremi- 
ties simply swinging to and fro, and that a flexible extremity having 
at least one joint is necessary to obtain such velocities. The extremi- 
ties act as a sort of “spring” which repels the animal from the 
ground. It was also pointed out that as a first approximation the 
action of only one pair of extremities, the rear ones, is essential, and 
that the existence of four pairs was probably determined by other 
considerations than speed. 

By very crude semi-kinematical considerations we derived a re- 
lation between the length of the extremity and maximum speed of the 
animal, and found that the latter should vary approximately as the 
square root of the length of the extremities. Quantitative data on the 
speed of animals are, to put it mildly, very scarce. Whatever data 
are available seem to bear out roughly the derived relation. 

The derivation of the above relation being in many respects rath- 
er unsatisfactory, it is the purpose of this paper to improve on the 
development of the theory. An even remotely exact theory is at pres- 
ent impossible because the equations of motion of a chain of linked 
levers (Fisher, 1906; Rashevsky, 1944c), as exemplified in an eXx- 
tremity of an animal, are of such a nature that the mathematician 


11 
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can only stare at them helplessly. Adequate methods of approxima- 
tion have not as yet been developed. We shall attempt to circumvent 
the difficulty by consideration of an over-all energy balance. We thus 
shall find a more complex relation between the speed of an animal 
and its size and shape, a relation of which the one previously derived 
may be considered as a first rough approximation. We shall also throw 
some light on the actual number of joints in an animal extremity, a 
question important in the theory of organic form. 

The mathematical treatment used in this paper will of necessity 
be very crude. We shall be more interested in orders of magnitude 
than in actual values, and in the general type of functional relations 
than in the absolute values of the coefficients of the functions in- 
volved. This must be kept in mind. 

Running is essentially a series of consecutive jumps. During each 
jump the animal is for a while completely without contact with the 
ground, its center of gravity describing a parabola during the “flight” 
phase. If at the end of each jump the total kinetic energy of the 
animal were completely lost, then the mechanism of running would 
be identical with the mechanism of jumping. Actually only a fraction 
of the total kinetic energy is lost at the end of each “flight” phase. 
Therefore the theory of continuous running is somewhat different 
from the theory of jumping. We shall first consider the latter. 

It is interesting to note that if the extremity were considered as 
a spring of original length J, contracted to a constant traction ae, 
and exercising a constant force F,, then, neglecting its mass, we would 
have the following relations. 

The total work done on expansion would be 


F(l—ol) = Fl(1—a). (1) 


Let M denote the mass of the body and v its velocity in the direction 
of expansion, that is, forming an angle ¢ with the vertical. Let g be 
the gravity acceleration. Expression (1) must be equal to the sum of 
the imparted kinetic energy 4$Mv? and the work done in lifting the 
center of gravity of the body by the amount Mg(1— «)1 cos ¢ . Hence, 


F(1—a)l=4Mv?= Mg(1—a)lcos¢. (2) 


Remembering that the horizontal component v, of v is equal to 
VU, = v sin ¢, we find from equation (2), 


| 20(1 — a) (F — Mg cos 4) 
n= | ees) aoe asst sin ¢; (3) 


that is, vm, « Vl. 
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If, as corresponds better to a real spring, the force F is equal to 
pal iG—y hc herey 


Poker | hie. 


where EF is the modulus of elasticity, then, putting (1 — a)l=za2, 
— ldo. = dx , we have for the work done by the spring, 


0 1 1 
i; Fax =— { (1—a)da 
(1-a)l E a 


l 
ip LCL a) 4.1 — a) J. 


(4) 


(5) 


This must be equal to 4Mv? + Mg(1— a) cos ¢. Hence, again, 
we find thatv « vl. 

Actually the extremity is a system of linked levers. We shall 
assume for simplicity that the thickness of the bony core of the ex- 
tremity is the same in all links, assuming the bones to be cylindrical. 
Let their thickness be 2b . 

The quantity b is approximately equal to the lever arm of the 
muscular force F which produces a torque bF in the joint. Again for 
simplicity, assume that the folding and extension of the extremity is 
complete; that is, that relative rotation of a link with respect to an 
adjacent one isa. Then the contraction of a muscle which produces 
such a relative contraction will be of the order of ab. Let the ex- 
tremity consist of n equal links, each of length //n. We shall num- 
ber both links and joints from top down. Thus the contraction of the 
i-th muscle acts on the i-th joint. Altogether, (n — 1) muscles must 
contract in order to rotate the joints relative to each other. More- 
over, a muscle must contract to rotate the upper joint with respect to 
the trunk of the animal. Hence the total number of muscles involved 
isn. 

We shall oversimplify our picture by considering the case where 
each link has only one flexor and one extensor muscle. Actually, the 
muscle which rotates the upper link with respect to the trunk is lo- 
cated alongside the upper link. We shall, however, consider the purely 
fictitious case where it is located in the trunk. In the further develop- 
ment of the theory a more realistic approach will have to be used. The 
present departure from the'real situation is justified, however, since 
we are at first interested primarily in some crude general relations 
which are not likely to depend much on the details of the structure. 
The assumption of equal links is in itself an abstraction. As in the 
early development of any theory, such abstractions are unavoidable. 


14 ON THE LOCOMOTION OF MAMMALS 


Assuming the muscular force F to be constant, the total work 
done in extending the extremity completely from a completely folded 
configuration is 

nF ban . (6) 


Very roughly, the force F may be set as proportional to the cross- 
section s,2 of the extensor muscle (Rashevsky, 1944b), where s, is the 
average thickness of the extensor muscle. Hence, denoting by f a fac- 
tor of proportionality, we have 


F=fs,?. (7) 


Since the relative contraction of any muscle does not exceed a 
fixed value, which is at most about 30% and is usually much less, 
therefore, in order to produce a contraction ba , the muscle must have 
a certain minimal length L. Denoting by m another factor of pro- 
portionality, we thus have L = mba, or 


eee pee: (8) 
ma man 
We have ex < 0.3 and probably 
en~ 0.1. (9) 
Since L = I/n, therefore 
b=el/n. (10). 


Hence the total work done during the extension is, by virtue of equa- 
tions (6), (7), and (10), 
W =afs,7el . (11) 


Denoting now by M the total mass of the animal, we find that 
this work is equal to the sum of 4Mv? of the kinetic energy of rota- 
tion of the links of the extremity, of the work in raising the center 
of gravity, and of the viscous work in muscles (A. V. Hill, 1927), 
which we shall consider for generality. If I denotes the moment of 
inertia of a link having the axis passing through its center of gravity, 
and w its angular velocity, then the kinetic energy of rotation of a 
link is 4Jw?. The kinetic energy of the displacement of the center of 
gravity of the links is contained in $Mv?. 

Assuming for simplicity an average total thickness s for both 
flexors and extensors, and considering, as an approximation, each link 
as an oblong ellipsonid of revolution with semiaxis 


a=U2n, c=b-+s,; (12) 
and putting the density p equal to one, we have (Appel, 1924) 


N. RASHEVSKY 15 


iE == mac? + act. (13) 

Hence, from equations (12) and (13), 
pa CO St oto]. (14) 

15 n An? 


Since, practically, b + s ~ 0-2 l/n, we may neglect the second 
term in the bracket and obtain 

Bye. (OSS) 2 

SOR ies 

: If we again denote by ¢ the angle made with the vertical by the 

direction in which the center of gravity of the whole animal is ac- 


tually propelled by the expansion of the extremities, and if we de- 
note by 


Ns (15) 


M,= M — Als? (16) 


the mass of the animal less the mass of the four extremities, then the 
work done in lifting the body during the propulsion is equal to 


W,=M,glcos¢. (17) 


It may perhaps be argued that since only two extremities out of 
four are actually involved in the propulsion, we should use, instead 
of equation (16), the expression M — 2ls?. This, however, is a detail 
which does not affect the final conclusions, and we shall therefore neg- 
lect it here. 

Considering the flexor and extensor muscles as approximating 
plastic, viscous, cylindrical rods of thickness s , and denoting by 7 the 
coefficient of viscosity, we have for the viscous force F, the expression 
a oh. 


—oo 


L dt 


If v, is the velocity of contraction of the extensor, then we have 
approximately 


(18) 


v 


—_=V,. (19) 


Hence, for each link, 


F,= Ves (20) 


During an interval of time dt, the displacement of the point of 
application of F, is v.dt. Hence the work done during dt is equal to 
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37870 2 
d (21) 
L 
The quantity v, is related to w, the angular velocity of the link, 
very roughly by 
Ye 00 (22) 


If t is the time necessary for a full extension of the extremity, 
then the average value of w is 
wo—n/2t. (23) 
Hence, 
=n2/2w. (24) 
From equations (21) and (22) we see that the total viscous work 
done during the time t in n links is approximately equal to 
3ns7v.2 an 3ys*b?wn 
W.= —_— = —_—__—_——., (25) 
L 2 2L 
If the extremity unfolds completely during the time ¢, then the 
average linear velocity imparted to the body is //t. During the same 
interval t each link rotates by an angle 2, so that the average angular 
velocity w is 2/t. Since we are interested here in general functional 
relations and in orders of magnitude only, we see that v is of the or- 
der of magnitude 


y=lo. (26) 
Hence, 
o=v/l, (27) 
The work W, as given by equation (11), must be equal to the sum 
of 4Mv?, 4 X 41nw?, the work W, and W,. Hence, 
W=4Mv? + 2Ine? + 4W,'+ W,. (28) 
Introducing equations (11), (8), (15), (27), (17), and (25) into 
equation (28), we find 
l(b '+ s)? 
afs,?el = 4Mv? + te els y2 
15n? 
+ 6as*ne?v + Mogl cos ¢. 


This is a quadratic equation in v. 
We shall first investigate the case where the frictional term is so 
small that it may be neglected. We then have 


____ | afels,? — Mygl cos ¢ 
oe Tay eo) 
4M + —————_ 


15n2 


(29) 
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If, as is usually the case (Rashevsky, 1944b), ¢ is large, the cos 
¢ term may be sed es Denoting by M, the mass of the extremity, 
remembering that /s? ~ M,., and that usually b << s, and putting 
8,7/s? =a, we find, ce equation (30) using equation (16), 


afeaM, 
4M, + [2 + (a/15n?) 1M. Gt) 


For animals which are geometrically similar, M@./M = const. and 
a =const. Therefore, for such animals the speed is the same. This is 
a more general form of a result obtained for a special case by A. V. 
Hill (1927) (c.f. also Rashevsky, 1944 b). In the absence of geomet- 
rical similarity, the speed increases with increasing mass of the ex- 
tremities, tending to the asymptotic value 


Umax = V1.4fae . (32) 


If, as has been made plausible in a previous paper (Rashevsky, 
1944a), we have s? « M,, and if a is constant, then instead of equa- 
tion (81) we obtain from equation (30), neglecting the cos ¢ term, 
and denoting by A and B two constants, 

eae 33 
y=. | ——. 
1+ Bl 

For not too large values of 1, we have v « \/l, which, as we have 
seen (Rashevsky, 1944 b), holds approximately. We find from equa- 
tion (33) 

Umax — VA/B. (34) 


Taking f ~ 107 dyn cm?, a ~ 1, and e ~ 0.03, and assuming 
M./M, ~ 0.03, as obtained from measurements of pictures of a dog 
whose speed was determined (Muybridge, 1887), we find from equa- 
tion (31) v ~ 2 X 10? em sec.. The actual speed is 12 X 10? cm sec."*. 
Since the data for f and « are very uncertain, no better agreement can 
be expected. 

All the above-discussed relations hold, however, only for the ini- 
tial jump. As has been remarked above, the loss of kinetic energy at 
the end of each successive jump is not complete, and therefore the 
theory should be modified for the case of continuous running. 

Practically only the vertical component of the velocity v is lost 
at the end of each jump. Hence in continuous running the extensors 
of the extremities must provide for each consecutive jump only the 


amount. 
4Mv? cos? ¢. (35) 
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Expression (35) should be used instead of the first term of the 
right side of equation (29). 

Another thing to be kept in mind is that after each jump the 
animal does not fold its extremities completely. 

Let us make the fairly plausible assumption that at the end of 
the flight phase the extremity, extended forward, also makes the angle 
® with the vertical (Figure 1). Since the loss of the vertical compo- 


FIGURE 1 


nent v cos ¢ of the velocity is not instantaneous, the upper part of 
the extremity will now move not in a horizontal line AA’, but will fall 
somewhat below it (Figure 1). The quantity h in Figure 1b repre- 
sents the sagging of the center of gravity of the animal during its 
contact with the ground. It also represents the amount by which the 
center of gravity must be lifted before each new flight phase, which 
begins when the extremity is in the position 1c. Defining the angle y 
as in Figure 1b, we see that 


h=l(cos y— cos ¢). (36) 


The circumstance that the extremity is now extended from only 
a partially flexed position also changes some of the expressions used 
above. Thus the contraction of each extensor is now not ab, but 
[x — (xn — 2w)]b = 2yb. Hence, instead of expressions (8) we now 
have 


ere rite (37) 
2my 2my 
Instead of (11) we have 
W = 2yfs,el , (38) 


and instead of (23) and (24) we have, correspondingly, 
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Be Nae (39) 


Making all those changes we now obtain, instead of equation (29) ; 
the following: 
al(b + 8)? 
= Se ed as V2 


2yfsel = LMv? cos? 6d + 
3 + ? 157? 


+ 12nyps?e?v + Mogl(cos 6 — cos y). (a 
Equation (40) determines the speed v, = v sin ¢ of continuous 
running for prescribed values of ¢ and y. What determines those 
two values? Let us first discuss the factors which determine ¢. 
When the body of the animal, or strictly speaking, its center of 
gravity, is projected during the flight phase with a velocity v at an 
angle ¢ with the vertical, the distance which it travels horizontally 
during the flight phase is given by 


2 


x=— sin ¢ COs ¢. (41) 
g 


The time it takes to travel that distance at the speed v, = v sin ¢ is 
given by 

x . 2p 

At=—=— cos ¢. (42) 

Vn g 
The larger the angle ¢, the greater for a given v is the horizontal ve- 
locity v,. The less also is the work of lifting the body just before the 
flight phase. Hence it is to the advantage of the animal to make ¢ as 
large as possible. An upper limit is, however, imposed upon ¢ by the 
following consideration. 

During the interval At the extremity must be folded and again 
extended forward, so as to pass from the position AO in Figure I to 
the position OA’. If ¢ is too large, then At, as seen from equation 
(42), may become too small to permit this change in the position of 
the extremity. 

The time it takes the extremity to change its position is deter- 
mined by the force developed by the flexors and extensors and by the 
inertial and frictional resistances. Let us first consider the inertial 
resistance. 

The work done by the muscular forces in folding the extremity 
is given by an expression similar to (11) in which, however, we sub- 
stitute s.2 for s,2, since now the flexors are active. The work of ex- 
tension is given by equation (11). The total work is given by 
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nfel(S.? + So”). (43) 
The kinetic energy is given by 4”Iw?, or, because of (15), by 
a B(b + 8)? a (44) 
60 n? : 
The angular velocity is therefore roughly determined by the equa- 
tion 
a B(b+s)? 
eee. 


o == Fifer ts, tsa ie (45) 
60 n? 


Since usually b << s and approximately s? = s,? + s,*, we have from 
(45), 


3 Nn? 
co? = 60 = fe. (46) 


For the time of the complete extension of the extremity we have 
expression (24). The folding time is of the same order of magnitude. 
Hence, roughly, 


At=—. (47) 
@ 


Introducing (46) into (47), the latter into (42), and solving for 
cos ¢, we find 
l 
cos ¢ = eee (48) 
2unv/ 60fe 

Using as plausible values v = 10° cm sec., f = 107 dyn em? 
(Rashevsky, 1946b.) ; «= 38 X 10°, 1 = 50 cm, we find cos ¢ ~ 10%, 
which makes ¢ almost equal to 90°. Actually, ¢, as observed from 
some photographs of running animals is about 75°-80°. Hence the in- 
ertial forces are not the limiting factor in determining ¢ , and we must 
conclude, in agreement with A. V. Hill, that the viscous forces are 
the main source of resistance in continuous running. Let us examine 
those. 

The work of the viscous forces in expanding the extremity is 
given by expression (25), which, because of (8), may also be written 
thus: 

327s?e?Lnw 
W.= re (49) 


But from (47) we have w =2/At. Therefore, and because of Ln = 1 ; 
we have 
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37° ns? E71 
W» = ree : (50) 


Introducing expression (42) into (50) and equating the latter 
to (48), we find 


(51) 


or 
4vf cos ¢ 
Das (52) 
omeg 

Now let us consider the factors which determine the angle y. If 
the viscous term in the right side of equation (40) outweighs all oth- 
ers, then y cancels out. This would mean that the speed of running 
is independent of y. Actually, although the viscosity term may be 
much larger than the others, those other terms probably cannot be 
completely neglected. In that case v is given by the positive root of 
the quadratic equation (40) and is a function of y, and of the other 

parameters, such asf ,s,,8,¢,M,7,n,1, etc. Thus we have 


v=F(y,f,5,8,b,¢6,¢,M,n,n,l). (53) 


We may now ask for the value of y which makes v a maximum. 
This value will be given by the equation 


—=0. (54) 


If such a maximum exists, then equation (54) determines yw in 
terms of the other parameters. 

In conclusion let us make a few remarks on what determines a 
number 7 of links. 

Since ¢ is rather large, the body of the animal is not lifted Bs 
much over its normal height above the ground. In moving the ex- 
tremity forward, it must be at least partially folded and then swung 
forward. For purely kinematic reasons, the clearance between body and 
ground must be larger than the length L = I/n of a link. That clear- 
ance is approximately of the order of magnitude of | cos ¢. Since, 
for actually observed values of ¢ (75°-80°), cos ¢ is of the order of 
0.25. Hence, approximately, I/n = 0.251, which gives n = 4. Ac- 
tually, due to the parabolic path of the body during the flight phase, 
the necessary clearance is somewhat larger than / cos ¢, which would 
reduce to about 3. These considerations throw some light on the fact 
that the extremities of the rapidly running animals have essentially 
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three links. Animals which, like the elephant or the rhinocerus, are 
more likely to walk fast rather than run, could kinematically afford a 
smaller number of links. Actually the tarsal part of the extremities 
in those animals is considerably reduced relative to the femural and 
tibial parts, which makes the extremity consist of only two major 
links. For rapid running, however, with a large value of ¢, less than 
three links would simply not permit the extremity to move forward 
during the flight phase. 

Another reason for such a structure of the extremity may be the 
following. If a chain of linked levers is to unfold in such a way that 
the end A is fixed and the end B follows a prescribed line, then for 
two links the position of one link prescribes the position of the other. 
But with three or more links this is not the case. Hence, for a two 
link extremity, the angular velocities of the two links would have to 
be exactly coordinated in order to extend the extremity in a given 
direction. But since, even for independent torques at the joints, the 
rotations of the links are not independent (Fisher, 1906), such a co- 
ordination would be possible, if at all, only for a very special choice 
of the different constants. In the case of three links this will not be 
so, as the system as a whole acquires much more flexibility. Little if 
anything is gained, however, by adding a fourth link. 

Another question is why the rear extremities of the animals, 
which act as the principal “propellers,” or rather, “repeilers,” are 
shaped as shown in Figure 2a rather than as in Figure 2b. It can 


readily be seen that in the latter case, at the moment when the ex- 
tremity separates from the ground, the lower link, rotating counter- 
clockwise, will produce a reaction of the ground in the opposite direc- 
tion to the direction of the locomotion, thus decreasing the efficiency 
of the locomotor mechanism (Rashevsky, 1946, pp. 91-92). 

All the above, while not constituting a rigorous proof, indicates 
the plausibility that the shaping of the extremities is determined by 
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requirements of mechanical functions compatible with the simplest 
structure (Rashevsky, 1943). 

If, as A. V. Hill (1927) thinks, and as seems to be borne out by 
the considerations on page 20 the most important factor in the re- 
sistance to locomotion is the muscular viscosity, then we may pre- 
serve only the third term in the right side of equation (40) and obtain 


fs,7l 
Cle ; 
678"e 


(55) 


If enough quantitative data of the desired type were available, 
we could, by comparison of those data with equations (30), (42), or 
the general equation which represents the solution of equation (29), 
draw definite conclusions on the somewhat controversial question of 
the magnitude of muscular viscosity. 

The author is indebted to Dr. H. D. Landahl for a critical discus- 
sion of the paper and correction of a few errors. 

This work was aided in part by a grant from the Dr. Wallace C. 
and Clara A. Abbott Memorial Fund of the University of Chicago. 
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Purely mechanical factors involved in the dynamics of the thin-walled, 
spherical viscus may be elucidated by studying the behavior of an arti- 
ficial viscus. An equation is developed on theoretical grounds which ex- 
presses the pressure-distention characteristics of such a model and appar- 
ently corrects the errors contained in similar equations developed by other 
investigators. A precise definition for wall “tone” is offered. On this 
basis the mathematics of “tone’”-distention relationships have been de- 
veloped. Experimental investigation of the artificial viscus (spherical 
rubber balloon) permits the differences in behavior between the ideal and 
the actual model to be composed, and confirms the validity of our theo- 
retical developments. 


Pressures developed within distended hollow viscera reflect not 
only the effects of well-recognized neuromuscular mechanisms, but also 
of certain poorly understood factors of a purely physical nature. Ex- 
perimental investigation and theoretical consideration of artificial 
hollow viscera have been of some value in separating the solely me- 
chanical factors involved in intralumen pressure production from 
those related to both the reflex mechanisms and the vital functions 
of living tissue. 

Kelling noted that inflation of a spherical rubber balloon pro- 
duced a rather rapid rise of pressure to a maximum, followed by a 
more gradual decline to a prolonged plateau of pressure. Shortly be- 
fore rupture of the balloon, the pressure dropped precipitously. Os- 
borne, conducting a similar type of investigation, obtained initial re- 
sults similar to Kelling’s. However, if he first “aged” his balloons by 
subjecting them to repeated inflations, he obtained pressure-disten- 
tion curves which rose slowly at first and then progressively more 

* This investigation was financially supported by USPHS grant RG 101. 


** The material presented here is part of a thesis for qualification for the de- 
gree of M.S. (Pharmacology). 
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rapidly. A series of infiations of excised dog bladders rendered a simi- 
lar type of pressure-distention curves. On a purely empiric basis he 
was able to fit hyperbolae to his curves with moderate success. Suther- 
land, applying the observations of Henry and of Goy that the strain 
produced in frog gastrocnemius muscle is a logarithmic function of 
applied stress, was able to obtain closer fitting curves for Osborne’s 
data. More recently Gianturco showed that the stomachs of intact 
cats can receive considerable quantities of material without a great 
rise of intralumen pressure (“receptive relaxation’). 

On purely theoretical grounds it should be possible to develop an 
equation which will accurately express the pressures developed with- 
in a hollow elastic sphere that obeys Hooke’s Law perfectly. Attempts 
to do so have lead to the following expressions: 


— 4d, fa fa 


P * ; (Modified from Winton and Bayliss) (1) 
T—To 
P=—4Ed, i , (Brody and Quigley) (2) 
r—Tp 
P=4Ed,r, ; (Frank) (3) 
Vs 


where P = pressure, FE = Young’s modulus, d) = wall thickness be- 
fore stretching occurs, r = radius, 75 = radius at which stretching 
begins to occur. ; 

Despite the obvious differences in these equations it can be seen 
that they belong to a family, 


ese"! 


P=42h dor"? : (4) 
n being an integer. 

The differences between equations (1), (2), and (3) are illus- 
trated in Figure 1. For small degrees of distention, r is approximate- 
ly equal to 7,. Therefore, the factor 7,""/r" from equation (4) is 
essentially independent of in this range, and the three curves are 
practically identical. When 7 is 1% greater than r,, the P of equa- 
tion (1) is only 2% greater than the P of equation (3). However, 
for greater degrees of distention, important differences occur. Taking 
the maximum value of P in equation (1) to be 100%, the maximum 
value in (2) will be 25%, and in (3) only 14.8%. Equation (1) at- 
tains its maximum at r= o, (2) atr = 2r, and (3) atr = 3/2”. 
Whereas equation (1) approaches its maximum asymptotically, (2) 
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mt AFTER WINTON & BAYLISS 


FIGURE 1. Graphic representation of various theoretical pressure-distention 
relations of the ideal thin-walled elastic sphere. AJ] three curves are drawn to the 
same scale. The upper curve approaches the top border asymptotically. Other 
maxima as indicated by vertical arrows. 


and (3) attain their maxima at the above finite values of distention 
following which they approach zero asymptotically. 

Reasons for the differences between equations (1), (2), and (8) 
are evident in their derivations: From the definition of Young’s modu- 
lus of elasticity H', the reactive tension 7’, per unit of cross sectional 
length of an elastic strip which has a thickness d, and has been 
stretched to a length 1, from a resting length J, , will be 


T = Ed(l— Is) /lo . (5) 


Applied to a lune cut from a thin walled elastic sphere, equation (5) 
becomes 


T = Emad (1 — 1) /7o. \ (6) 


Enan iS the “manifest” modulus of elasticity for a sheet of material 
which is stretched equally in all directions. Our theoretical develop- 
ments show that Hman = 2H, a finding which is in agreement with 
Frank. Thus equation (6) becomes 


T=2Ed(r—1)/%%- (7) 


Substitution of equation (7) in the well-known formula for pressures 
developed within a hollow sphere, P = 2T/r (cf. Winton and Bayliss, 
p. 305), yields 
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Ed one 0 
fees harite Re: (8) 


To vi 


Since the wall of an elastic sphere obviously thins on distention, 
d of equation (7) is a variable. Winton and Bayliss overlook this fact 
and imply that d = d., dy being the wall thickness before stretching 
occurs. Frank notes that wall thickness is inversely proportional to 
the square of the radius and therefore assumes d = (107/77) d). We, 
on the other hand, emphasize the fact that although true wall thick- 
ness varies in the inverse quadratic manner described by Frank, the 
actual number of elastic fibrils per unit length is inversely propor- 
tional to the first power of the radius. For this reason we employ 
the concept of an “effective wall thickness” characterized by d = 
(7,./r)do. Substitution of each of these values of d in equation (8) 
yields equations (1), (2) and (8). 

If the radius and intralumen pressure of a thin-walled elastic 
sphere are known, the wall tension may readily be calculated from 
P= 2T/r. However, it is our opinion that wall tension so calculated 
is not equivalent to wall “tone” inasmuch as the number of elastic 
fibrils per unitvof length varies with distention. 

According to our concept, tone is not the tension exerted by a 
unit of wall length but rather the tension exerted by a given number 
of the fibrils which comprise the wall. A convenient number of fibrils 
to employ as a reference are those contained in one centimeter of un- 
stretched wall (cm,). Therefore, in the experimental work to be de- 
scribed later in this paper, tone is expressed as gm/cm,. Such values 
are determined by multiplying the experimentally determined values 
of T by the corresponding linear distention factors, r/r,. Multipli- 
cation of equation (7) by the factor r/r,, and substitution of d = 
(r,/r)d) (our concept of “effective wall thickness”) yields 


Tone = 2Ed,(r — 1.) /%o . (9) 


Equation (9) is of the same form as equation (7) except that the 
quantity d, enters now instead of d. It should be noted that this is a 
linear equation, 2/'d,/r. being the slope and 7, the r intersect. Simi- 
lar operations on equation (7), employing d = d, (implied by Win- 
‘ton and Bayliss) or d= (7,?/7?)d) (Frank) do not lead to linear func- 
tions. Since tone represents the pull exerted by a constant number 
of fibrils, we believe that the linear relation to strain derived by us 
is correct. 

In an effort to test the validity of our equations we conducted a 
‘series of balloon inflation experiments in a manner similar to that 
described by Osborne. Tension was experimentally determined by 
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substituting measured values of pressure and radius in the equation 
P = 2T/r. “Tone” was then calculated by multiplying each value of 
tension by a corresponding linear distention factor, 7/1o. 

Figure 2 represents the results of a typical experiment. Although 
the “tone” curve is not a straight line because rubber is not a per- 
fectly elastic material, as a rough approximation we may represent 
each of the first two portions of that curve by a segment of straight 
line. Straight lines were graphically fitted to these portions and ex- 
tended until they intersected the abscissa. The two points of inter- 
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FIGURE 2. Correlation between theoretical and actual pressures produced 
by distention of an artificial thin-walled, spherical viscus (rubber balloon). Solid 
portions of the straight lines and pressure curves indicate the regions where 
our equations (2) and (9) in the text may be applied to the experimental data. 


section constitute respectively 7) and 7)’. The slopes are equivalent 
respectively to 2Ed,/r, and 2E'd,’/7, (Cf. equation 9). Wall thick- 
ness before stretching, d,, being known, values of Young’s modulus 
(EZ and E’) may be calculated from the slopes of the two portions un- 
der consideration. However in the case of the second portion of lin- 
earity the correction, do’ = (70/7) do , must be made. The next step 
in curve fitting involves calculation of 4E'd, and 4E’d,’ from the two 
slopes and 7 and 7’ respectively. Substitution of these various fac- 
tors in equation (2) gives two equations which agree fairly well with 
the observed pressures in the regions where they are applied. 

This method of reducing data also permits calculation of Young’s 
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modulus in regions where it is variable (d Tone/dr # constant) by 
graphically fitting tangents to the “tone” curve at various points 
under consideration and treating them in the manner outlined in the 
preceding paragraph. It is probable that the increasing slope of the 
“tone” curve beyond the second linear portion is due in part to re- 
orientation of elastic fibrils in the directions of stress. However, since 
the stress is uniform in all directions of the surface of the balloon, 
the errors arising from this source in our experiments must be con- 
siderably less than those which would result from the application of 
one dimensional stress ordinarily employed in investigating elastic 
properties. 

Hollow viscera are neither spherical nor perfectly homogenous, 
and in their in vivo situation factors other than contraction of their 
wall musculature contribute to intralumen pressure production. How- 
ever such organs as heart, stomach, and bladder, particularly when 
distended, approximate the spherical contour sufficiently to warrant 
some application of the foregoing mathematical developments to their 
mechanics. In our hands, interpretation of gastric pressure-filling 
curves in the light of equation (2) has proved valuable, and we have 
been able to demonstrate important physical factors in the phenome- 
non of “receptive relaxation” (Brody and Quigley). 
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This paper describes a mechanism for (1) bettle-neck optic nerve 
conduction and (2) the recognition of visual spatial forms. (1) In this 
mechanism the w retinal receptor neurons are in one-one causal relation 
to a set D of w central neurons, the connecting optic nerve containing 
less than w fibers. (2) Also a set ¢ of u spatial forms (herein defined) is 
in the one-one causal relation to a set F' of » central neurons so that if 
f, (any given number of F’) fires, then the corresponding ¢, has occurred 


in the retinal image. Only ¢, can fire f,, but f, may fire for any position, 
size or orientation of ¢, on the retina, with the restriction that ¢, not be 


smaller than a minimum size which is a function of retinal position. 
Neuron economy is considered throughout. 


I. Hypothetical Retina, Optic Nerve and Central Retina 


For convenience we will arbitrarily assume that for some choice 
of the time t = 0, no neuron fires at non-integral values of ¢ in the 
nerve net.discussed. The interval from the start of a “time of action” 
in any neuron to the end of the succeeding synaptic delay at any axon 
synapse of that neuron is assumed constant and equal to .0005 sec., 
and this interval 6 is taken as the unit of time. (McCulloch & Pitts, 
1948, Rashevsky, 1946; p. 30). We shall also follow the notations of 
those papers. The & and z refer to the logical sum and product ex- 
cept where the context indicates otherwise. 

There are about 137 million receptors in one retina but fewer 
receptor neurons, as most of the latter extend to a plurality of recep- 
tors. In the hypothetical monocular network described below we will 
take the number w of retinal receptor neurons as 25 million and as- 
sume that each has just one receptor. We describe first the neural con- 
nections whereby these w photoreceptors, illustrated by set A in Fig- 
ure 1, are in one-one causal relation to the similarly illustrated cen- 

tral set C of w neurons. (See also H. D. Landahl, 1939). 
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FiIcuRE 1. For correspondence with the text the following changes should be 
made in this figure. Neuron sets labeled z, , z,, 23, °° 2, °** 2, should be labeled 2z?, 
a, z3,--- gl --- 2”, In the large cycle there should be three internuncials between 
«“, and «,, between x, and x,, between a, and x,, etc., and finally between m9 
and €m,» and between Cm, and «#,. There should be two internuncials between 
Band x,. 

In the cycle between levels C and D the impulses run clockwise. Although the 
connecting line is omitted, dendrite b;» is part of the neuron Vi- 


a2 
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Set A is divided into m, equal subsets 
A1(@,1, dp) ++- G2 --- Qn?) 
A? (a,?; a,” eee a;? ee Gn? ) 


A*(G,*, dy* +++ @;* +++ On*) 


No Mo Mo... a.m am 
A (a, ? Cy Qi pases An) 


of %) neurons each, so that a;* denotes the ith member of the Ath sub- 
set A*. Thus m2 = w, and as a particular case we will take n) = 10° 
and m, = 25. Neuron set A is in one-one relation to a similarly sub- 
divided set B of w dendrites so that a;* has one end bulb on b;* whose 
threshold is 2. The retina contains also a cycle of km, neurons where 
k = 4 is the refractory period for neurons in the net. (Since k = 4 
the maximum possible discharge frequency is 500/sec.). The first 
neuron in the cycle is designated by $ and the (kA)th neuron by a. 
Neuron x, has one end bulb on 6;* so that a,;*(t—1) - x. (#1) -—>- 
b;*(t) which is read, “If a,* fires at t—1 and x, fires at t—1 then b;* 
fires at t.”’ The arrow (—) is used as the implication symbol instead 
of the horseshoe. 

In the inactive cycle let just 6 fire for any reason at some time 
(so as to start the cycle) and thereafter let q(t) -=-p(t—1) where p 
and q are consecutive cycle neurons and precedes gq. Thus only one 
impulse is passing through the cycle. Choosing any time at which £ 
fires as t = 0 we have 


B(0), (1) 

B(lkm); 1=0,1,2,---[kK=4,m= 25], (2) 
heli ok) s (3) 

¥.(¢) -=-t—=k(A Elm) —1. (4) 


The period of the cyclical impulse is km,.d. Receptor neuron 4@,* is 
associated with K—1 neurons as shown in Figure 2, giving 


b(t) -=-m(t—1) -Sar(t—f), (5) 


j=1 
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where > indicates the logical sum of the K propositions. We will take 
K=4, 


| ps 


s) 


tz 
2 » 
4} 2: 
FIGURE 2 


The optic nerve contains n) + m» neurons. There are % = 108 


neurons each designated v;(i = 1, 2,38, -:- %). Neuron v; has the 
m, dendrites b;1, b;2,--- b;+ --- biv”. There are m,) neurons each desig- 
nated «(4 =1,2,3,---m,). Cycle neuron x, has an end bulb on 2’. 
Thus 
™o 
vt) -=->b(), (6) 
A=1 
x(t) -=-a(¢—1). (7) 


Proposition (6) merely states that v; fires whenever any one of its 
Mm, dendrites fires. 

Let C be a set of w central neurons (a “central retina’) in one- 
one relation to A and likewise divided into its corresponding m, sub- 
sets of mn) neurons each. Neurons v; and 2* each have one end bulb on 
c;* and the threshold of c;* is 2; therefore 


c(t) -=-v;(t—1) -a2(¢—1). (8) 


Thus if c;* fires at t then a,* fired at either t—2, t—3, t—4, or (5, 
which may be written 


K+1 
en(t) > Dar (t—J). (9) 
j=2 
This gives complete specificity (spatially) for the central discharges, 
no member of the central retina firing unless the corresponding mem- 
ber of the retina fires. 

We see that a;* may fire and not cause ¢;* to fire since att =k 
only members of B' may fire, at t = 2k only members of B? may fire, 
etc., so that members of B* may fire only once every km,6 . If a,> fires 
once the probability p that this will cause ¢;* to fire is K/km, . Normal- 
ly, however, any effective stimulus gives rise to a volley of impulses 
from each receptor affected. If a;* fires a volley of impulses the prob- 
ability P that this volley will cause ¢;* to fire is, of course, greater than 


J. T. CULBERTSON 35 


p. Let f, be the average discharge frequency of a;* during the first 77,6 
after the volley starts, that is, the number of firings during the first 
km,6 divided by km. Also let f, be the average frequency during the 
second km,6 , etc., and finally let fy be the average frequency during 
the last km,.6 after the volley starts and in which there is some mem- 


M 
ber of the volley. Then P=1— TI (1 — Kfi) , if we assume that Kf; 
is the probability that some member of the volley during the ith km6 
causes c;* to fire. 

Since the subsets of A send impulses through the optic nerve suc- 
cessively the latter contains n) +m. = 1,000,025 fibers, but if the sub- 
sets sent their impulses through simultaneously the nerve would con- 
tain 1%) = 25,000,000 fibers. S. L. Polyak (1941, p. 447) estimates 
about 10° fibers in the optic nerve. 

Using the above mechanism with w = 2.5 X 107 we see that the 
number WN of optic nerve fibers has the minimum value of 10‘, since 
minimizing N = m + wm, we get dN/dm = 1— wm,? = 0 so that 
Mo = Ny = Vw and Nmin = 2 \/w = 10,000. As N decreases, however, 
(from m, — 1) the time T = km,6 for the passage of impulses from 
all the subsets increases. In the net described T — .05 sec, and if N 
was a minimum we would have T — 10 sec. 

In C the m, subsets fire in succession, each firing once every 
km,6 , and no two firing at the same time. To get the neuron set D 
in which all m, subsets fire simultaneously let D, the “synchronized 
retina”, consist of w neurons in one-one causal relation to C so that 
¢;* is connected to d;* by a cycle of & neurons with an inhibitory bulb 
from x* as shown in Figure 1. Thus in D all m) subsets fire simul- 
taneously every ké, and if c;* fires at t then d;* fires at the m, times 
t+2,t¢t+6,t+10,---¢ + (m-—1)k + 2 and conversely. This may 


be expressed by 
Mo-1 
oA(t) -=-Hd)(t+ 74 2). (10) 
j=0 
The firings in D are in one-one causa] relation to the firings in A so 


that there is complete specificity (spatially) for these central dis- 
charges, no member of D firing unless the corresponding member of 


the retina fires. Thus we have 
Mok 
d;(t) ->-Sa)(t¢—j—4). (11) 
j=0 


The nbove mechanism for optic nerve conduction and central repre- 
sentation leads directly to the mechanism for form recognition de- 
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scribed below. The retinal impulses are “funneled” successively 
through the one set v in the optic nerve. Now in somewhat the same 
way impulses in D “funnel” successively through a set z of central 
neurons. In this second mechanism, however, there is not just the 
one set z, but many such sets 21, 22, --- 2’, --- 2”, (Figure 1) so that 
many such funneling processes are occurring at the same time. When 
impulses “funnel” successively the same term will be used for the 
pathways over which they funnel; so we will say, for instance, that 
the m, subsets of A “funnel” successively through the set v, and we 
will use this term whenever sets of neurons are related in this way. 
It is understood that if m sets funnel through one set, there is an 
associated cycle with firing intervals as described above. 


II. Hypothetical Mechanism for the Reproduction of Forms 


Henceforth for simplicity let the w (= 2.5 X 10%) receptors be 
equally spaced in a square array of rows and columns. Leaving the 
mechanism described in section J unaltered let the “synchronized reti- 
na” D be geometrically similar to the retina, the cell body of d;* be- 
ing in the same relative position as the receptor of a;*. This is for 
simplicity of explanation only since the cells of D may have any rela- 
tive positions whatever. 

To illustrate first reproduction involving translation only, choose 
any m square areas within D (the sides of each square being parallel 
and perpendicular to the sides of D) containing n = 225 neurons and 
let the m sets s', s?, --- st... s” inclosed by these squares funnel suc- 
cessively through a similar square set z of n cells after D. Denote 
by si; the member of the ith row and jth column in each of these m 
sets, use a similar notation for z, and let s;;* have one end bulb on 2;; 
so that 


2ij(t) «= Ssyk(t—1) -ax(t—1), (12) 


where x is the cycle neuron corresponding to s*. Here T = mk. Su- 
perimpose any solid figure, say a T on z and let the ow cells inside it 
each have an excitatory bulb on some neuron h whose threshold is w 
and each cell outside have an absolute inhibitory bulb on h. (Less 
strict conditions for the firing of h are given below.) Neuron h fires 
at t + 1 for only this one firing pattern (call it ‘- ) inzatt. Define 


the activity matrix P(z, t) as follows. Pattern P(z, t) is a square 
matrix of n = 15? elements aj; in which [a;;] = 1 if 2i3(t) is true 
and ai; = 0 if zi;(t) is false. (Landahl and Runge, 1936; Landahl, 
1947.). The neuron h fires at ¢ + 1 if and only if yal (2 9 = P.. 
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Because of the above funneling it follows that any given value of t, 
P(z,t) = P(s*, t—1) for some value of x. If h fires at t + 1 then at 
t—1 some one of m different patterns in D were identical to P.. Thus 


there are m different positions on the retina in which an oe of T 
A proper size and orientation) may be projected so as to cause h to 
re. 

Let the square area on D inclosing s* be divided into 1 equal 
small squares and let there be one member of s* in each small square. 
This large square area divided into smaller squares will be called a 
mapping. In general a mapping is any square area on D divided into 
n = 225 equal small squares such that at least one member of D is in 
each small square. In Figure 3 one mapping is illustrated and 7 is 


FIGURE 3 


taken as 25 in order to make a simpler diagram. To illustrate the 
general reproduction of forms, replace the m special mappings of the 
preceding paragraph by any m mappings M1, M?... M*.-- M™ differ- 
ing generally in size, position and orientation, and let the m sets 
S1, 82, --- st .-- s™ each consist of all neurons within one mapping, i.e., 
s* consists of all neurons within M*. As mappings may overlap, some 
neurons in D may be in more than one of these m sets. Designating 
arbitrarily one edge of each mapping as the top and then numbering 
rows and columns in the conventional way (mirrored mappings are 
excluded), any small square in M* is in a numbered row and num- 
bered column. In M* each neurow in the square in the ith row and jth 
column is designated s;;*. Each of these neurons i.e., each s;;* has one 
end bulb at a synapse on z;;. At that synapse there is one end bulb 
from each s;;* and no other end bulbs, and the threshold equals 1/2 
the number of these end bulbs. Hence this synapse fires at ¢ if and 
only if at least half of the s;;"’s fire at {—1. There are, of course, m 
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such synapses (functionally separate collections of end bulbs) on Zi; , 
say on m different dendrites of z:;. Then P(s*, t) ie., the pattern or 
activity matrix of s* at t is defined as a square matrix of n = 15’ ele- 
ments in which a;; = 1 if at least half of the s;;*’s fire at t and ai; = 0 
if less than half fire at t. For any value of ¢ , just as before, P(z, t) 
= P(s*,t—1) for some value of x . 

Thus there are m different variations of position, size, rotation 
on the retina in which an image of a T may be projected so as to cause 
h to fire. 

In the hypothetical mechanism described below there is not just 
one funnel through one set z but » = 100 funnels through the » sets 
gi, g2,-+-gl.-.2” (see Figure 1). We have m = 640, 000 and this is the 
same for all the » funnels, so that T = 1280 sec. for all funnels. The 
value of T will seem absurdly large especially in regard to the short 
reaction times in form recognition, until it is shown later how the 
entire cycle need not be run through, but instead the impulse may 
pass through part of the cycle and then jump to another part of the 
cycle in such a way as to give short reaction times. The accessory 
“attention” mechanism to omit parts of the cycle is described later. 
Also, the large value of m makes the number of synapses on 2;; too 
large for the anatomical evidence, but this is easily corrected later 
after the main outline of the mechanism has been given. 

For simplicity all the » sets at level Z (see Figure 1) are identical 
in structure, and the » sets at level H are identical in structure and 
also in the contacts they make with the Z level. Only one neuron in 
each set at level H is shown. 


The total number NX of mappings which give the sets s in the 
mechanism is N = ym = 6.4 X 10’. Suppose at the H level some neu- 
ron u has contacts so as to fire at t + 1 if and only if P(z’, t) = age 


Then there would be a set X of » = 100 similar neurons, one above 
each z', such that there are 6.4 X 10’ different variations of position, 
size, orientation on the retina in which an image of a T may be pro- 
jected so as to cause some member of X to fire. No member of X fires 
for any P(z,t) which does not equal Pie Level F is an optional addi- 


tion to the mechanism such that some one f fires if and only if any 
member of X fires. If we include level F in the nerve net, then there 
are N variations in which the image T may be projected upon the 
retina so that the one neuron f fires. No other image can cause f to 
fire, and no other neuron at level F fires for that image. 

The sets at level 7 are identical and the sets at level H are iden- 
tical but each set, s, from any mapping passes through just one of 
the y funnels. If we include level F in the nerve net, then there are 
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XN variations in which the image T may be projected upon the retina 
so that the one neuron f fires. No other image can cause f to fire, 
and no other neuron at level F fires for that image. 

The sets at level Z are identical and the sets at level H are iden- 
tical, but each set s from any mapping passes through just one of the 
» funnels. 

The particular value of XN given above depends upon a systematic 
way in which the mappings are applied to D. All numerical values 
stated are arbitrary but illustrate the requirements that every part 
of D be covered by at least one mapping, that the area of the smallest 
mapping covering any point in D is greater for points near the periph- 
ery than for points near the center, and that the whole range of varia- 
tions in area, position, orientation be filled in as uniformly as possible. 

That edge of D which is causally related to the upper edge of 
the visual field will be called the top of D. Consider a mapping whose 
top is parallel to the top of D. Furthermore, the top of that mapping 
is nearer to the top of D than the bottom of that mapping. Such a 
mapping we shall call upright. There are m = N/y of these as follows. 

The smallest upright mappings each have an area of 225, taking 
the distance between two consecutive neurons in a row or column of 
D as the unit of distance. Multiplying by 1.1 we get the area of the 
next largest mappings, and multiplying this again by 1.1 we get the 
area of the next largest, etc. In this way the area of any upright map- 
ping is 225 (1.1)’ where y—0,1, 2,--- 121,122 as y = 122 gives the 
largest possible upright mapping, so we have upright mappings of 
123 different sizes. The center squares of upright mappings having 
the same area do not overlap but fill a square area and the center of 
this square area is the center of D. 

Let Ny, be the number of upright mappings of area 225 (1.1) 
and let N, be a maximum such that 


RS Ny = (10+ y)? 


where FR is the greatest number of upright mappings of area 225 
(1.1)7 that can be mapped on D so that their center squares do not 
overlap. Then for each value of y < 81 there are (10 + y)? map- 


81 

pings whose area is 225 (1.1)? so that 5 Ny = 10? + 11? + 12? +--+. + 
y=0 

912 = (91-92-183-10-11-21) /6 = 255,001 mappings. For y > 81 the 


122 
values of Ny decrease somewhat irregularly and we have YN, = 87° 
y=82 


4+ 82? + 78? + 742 + 69? + 67? + 62? + 59% + 55? + 52? + 497 + 
46? + 43? + 40? + 38? + 35? + 83? + 312 + 29% + 27? + 25% + 227 + 
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212 + 20? + 192 + 172 + 162 + 1424+ 187 +122?4+1274+9° +947 
+ 62+ 52+ 52+ 42 + 32 + 2? + 1= 66,207. For the total of all upright 
mappings we then have SN, = 321208 = 3.2 X 10°. We get all the 
mappings, upright and non-upright, by rotating each upright mapping 
through the » = 100 angles 2in/y (l= 1, 2,--- v) so that the total 
number of these mappings is 3.2 X 107 Any such orientation of an up- 
right mapping is defined as giving a mapping even though some small 
squares of the rotated area may fall entirely outside D. Mappings at 
an angle .02ix give the sets which funnel through z’. Thus the sets of 
all mappings at 3.6° funnel through z’, the sets of all mappings at 7.2° 
funnel through z?, etc., and finally the sets of all upright mappings 
funnel through z’="™. 

The above mappings are half the total number N. The other 
mappings are made on D’, a manifold exactly similar to D. We add 
w neurons at the same level as D to form D’. Each neuron having x 
end feet on any neuron in D has x end feet on the corresponding neu- 
ron in D’ so that P(D , t)-= P(D’, t). For each mapping M on D we 
derive the mapping M’ on D’ by translating M so that its center point 
is moved \/2/2(1.1)7 units to the top right corner of its center square 
83,3. Remove this mapping from D to the same position on D’ to give 
the mapping M’. Thus N=235 N,. 

We will suppose conditions arranged so that only T shaped images 
are projected on the retina one at a time, and that each is projected for 
the time interval T — mk. Ignoring end organ adaptation we will 
suppose that maximum discharge frequencies always occur and con- 
tinue throughout the time of stimulation. We wish to show that (1) 
certain small variations of any T image which causes neuron f to fire 
will also cause f to fire; then more generally that (2) continuous 
variations of the image will cause f to fire. This description of the 
mechanism will be continued in a subsequent paper. 

This work was aided by a grant from the Dr. Wallace C. and 
Clara A. Abbott Memorial Fund of the University of Chicago. 
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A “probabilistic” rather than a “deterministic” approach to the 
theory of neural nets is developed. Neural nets are characterized by cer- 
tain parameters which give the probability distributions of different 
kinds of synaptic connections throughout the net. Given a “state” of the 
net (i.e., the distribution of firing neurons) at a given moment, an equa- 
tion for the state at the next moment of quantized time is deduced. Cer- 
tain very special cases involving constant distributions are solved. A 
necessary condition for a steady state is deduced in terms of an integral 
equation, in general non-linear. 


ib 

It is becoming increasingly evident to those engaged in develop- 
ing the “neural net” approach to psychology that the methods used 
in the early phases of development of the theory will need to be sup- 
plemented in future work by a theory taking greater cognizance of 
the anatomy of organisms. 

This need arises largely from the following shortcomings of 
specifically constructed neural nets: ; 


1. It can be shown that for any postulated neural net, whose 
“firing pattern” is isomorphic to the invariant aspects of a given psy- 
chological phenomenon, there can be constructed an infinity of alter- 
native neural nets isomorphic to the same phenomenon. 

2. Some of the neural nets require a greater number of neurons 
in their structure than is contained in the entire nervous system. 

3. Most of the neural nets thus far considered are so highly 
specific in their structure that the failure of even one neuron to func- 
tion renders the entire mechanism useless. 


The first of the above considerations raises the problem of de- 
termining which neural nets (of an infinity of alternatives) are in- 
deed present and functional in the nervous system of the organism. 
The answer to this problem probably rests upon the invention of new 
laboratory techniques which will give a practicable procedure of 
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tracing directly the neural circuits. Such techniques, however, will 
probably be aided largely by less direct inferential methods such as 
those, for example, which are currently employed by the physiologi- 
cal psychologists. 

The second and third considerations are more serious weakness- 
es in the specific net work theory. The second clearly rules out some 
of the postulated neural nets, while the third raises an uneasy ques- 
tion, namely, can we expect that the genetic constitution of a zygote 
carries a sufficient amount of “information” to predetermine the 
growth of a highly complex nervous system consisting of a tremen- 
dous number of minutely specific neural structures? 

It would be meaningless to say that this is either probable or im- 
probable since any statement about the probability of an event re- 
quires a knowledge not only of the event itself but of all its alterna- 
tives. Since such knowledge is lacking, the authors will be content to 
express a mere suspicion of its improbability. The suspicion is found- 
ed on the following evidence: 

1. Lasheley’s experiments on rats (Lasheley, 1926) demonstrat- 
ed that the maze learning ability does not depend upon any specific 
region of the cortex but is roughly proportional to the amount of cor- 
tical tissue possessed by the animal. These experiments tend to refute 
the “specific connection” hypothesis at least in those animals. 

2. Identical twins of the same genetic constitution and remark- 
ably similar macroscopically differ on the microscopic or the near 
microscopic level, for example, in their fingerprints (Newman, 1940). 
This evidence suggests that the genes may not predetermine the de- 
tails of microscopic structure but may exercise only a “statistical in- 
fluence”, (McCulloch & Pitts, 1947) a “bias” on the growth of tissue. 

3. From the point of view of the “specific connection” hypothe- 
sis, bearing in mind the mechanisms of inheritance, one would ex- 
pect that a great many psychological traits would either be present 
in an organism or entirely lacking. In reality organisms are found 
to possess traits in almost any degree of “intensity” or “facility”. 

A statistical approach to the structure of the nervous system 
may be able to cope with these difficulties. Two distinct avenues of 
development are suggested: (1) a statistical ‘description of the evo- 
lution of the nervous system and, (2) a statistical description of the 
functioning of a neural net. The authors, after considering some 
questions arising in both types of approach, have decided to start 
with the latter. Starting with “completely random” structures with 
only the crudest properties of neural nets, it is the intention of the 
authors to obtain “order out of chaos” (as N. Wiener once said it). 
If we are fortunate to be able to describe some overall properties of 
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nets which lead to a type of functioning resembling actual nervous 
systems, we shall then attempt to formulate the growth biases which 
lead to the formation of such nets. In this way it is hoped that the 
two approaches may become unified. 


We are thus led to study nets which exhibit certain properties 
by virtue of “tendencies” and “biases” in their connections and struc- 
ture. In classical physics an analogous task was undertaken in the 
study of the behavior of gases. The kinetic theory has greatly eluci- 
dated the problem by its use of “parameters” such as density, veloc- 
ity distribution, etc., which obviate the need of taking into account 
the disposition of each molecule of the gas. In a similar way we pro- 
pose to use parameters which will describe only “gross” properties 
of a neural net. 


Te 

In order to facilitate the mathematical symbolization of the prob- 
lem and to illustrate the approach with a specific formulation, it will 
be necessary for us to define some components of nervous tissue in 
a highly schematized way, taking into account only those aspects of 
the components which are pertinent to our formulation. Thus, al- 
though dignified by a biological nomenclature, our components will 
be frankly mathematical constructs stripped of all properties except 
those specified in the following definitions. 

Neuron: A point in space characterized by one of two alterna- 
tive states, namely, firing or not firing, and by an integer called the 
threshold. 

Bulb: A point in a region immediately adjacent to a neuron, char- 
acterized by one of two alternative states, namely, active or not active. 

Azone: A relation between a neuron and a set of bulbs, such 
that if the neuron is firing, all of the bulbs of the set are active at 
the same time. The bulbs associated with a neuron in this way will 
be said to originate at that neuron. They will also be called the termi- 
nal bulbs of the axone. 

Synapse: The region immediately adjacent to a neuron charac- 
terized by a finite number of bulbs, and a certain partition of the 
bulbs into “bulb groups”. There is thus a one-one correspondence 
between synapses and neurons. All the bulbs of a synapse will be 
said to terminate on the neuron associated with the synapse. 

Bulb group: A set of bulbs in the same synapse originating at 
the same neuron. Depending on the number of bulbs in a bulb group, 
the latter will be referred to as a “single”, a “couple”, an “n-tuple”, 


etc. 
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Neural net: The totality of neurons, axones, and synapses con- 
sidered and their distribution in space. 

Synaptic type: Two neurons will be said to be of the same synap- 
tic type if their synapses are characterized by the same number of 
bulbs and the bulbs are partitioned into bulb groups in the same way. 

The following postulates will govern the present formulation. It 
is the authors’ intention to complicate these simple postulates in suc- 
ceeding formulations so as to approach more plausible representa- 
tions of actual biological structures. 


1. If a neuron fires, all the terminal bulbs of the axone asso- 
ciated with it are active at the same time. 

This assumption was already stated in the definition of the axone. 
The postulate supposes instantaneous propagation of impulses along 
the axones. The same hypothesis governs the McCulloch-Pitts for- 
mulation, where it is justified by the fact that for short distances syn- 
aptic delay is great compared with the time of propagation along the 
axone. We will suppose this to hold in the regions considered. 

2. If and only if a number of bulbs at a synapse, not less than 
the threshold of the associated neuron are active at an instant t, 
the neuron will fire at the instantt +1. 


This implies a constant “synaptic delay” for all synapses, which 
is taken as the unit of time. In the present formulation we shall sup- 
pose a “state” for a given neural net at the time t = 0 and will seek 
the state of the net for succeeding moments, all integral multiples 
of our unit. 

Certain important concepts which form indispensible features 
of theories dealing with nervous activity are here deliberately 
omitted. We have already mentioned that the time of propagation 
of an impulse along an axone is neglected. Hence the “length” of an 
axone has no meaning. Distances between neurons, however, will ap- 
pear in connection with other considerations. The refractory period 
of a neuron (i.e., the time interval during which it cannot fire, im- 
mediately following the time of firing) is likewise omitted from con- 
sideration. The omission is justified by our assumptions, if the re- 
fractory period is taken smaller than the synaptic delay, since in this 
case a neuron may fire at two successive moments of our “quantized” 
time. Finally the omission of inhibitory connections from considera- 
tion is conspicuous. These will be included in a future formulation 
of the problem. 

Our neuron, then, is characterized by three variables, namely, 
the number of bulbs terminating on it, the way these bulbs are parti- 
tioned into bulb groups, and its threshold. The first and third of 
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these variables are clearly integers. The second depends on a certain 
partition of an integer. 

Let us suppose that six bulbs terminate on the neuron and that 
they are grouped into two couples and two singles. Then the parti- 
tion of 6 associated with our synapse is 1 + 1+2+2. There are 11 
distinct partitions of 6. Hence we may have 11 different kinds of 
neurons, each having 6 bulbs terminating on it and each having the 
same threshold. In general if the maximum number of bulbs at any 
Synapse of a neural net is N , and the greatest threshold of any neu- 
ron is H , the net may have Np(N)H kinds of neurons, where p(N) 
is the well known function which gives the number of partitions of 
an. integer. 

All the partitions of an integer can be ordered according to some 
prescribed system. For example, a partition of the integer n can be 
symbolized by an n-tuple( i,, %, --- %,) of non-negative integers, 
where 7, indicates the number of 1’s in the partition, 7, , the number 


of 2’s, etc., so that n= S si;.. At least some of the 7,, of course, must 
s=1 

be zero. These n-tuples can then be ordered in some prescribed way, 
for example lexicographically, that is in such a way that the n-tuple 
(4, , 2, -*+ tn) precedes the n-tuple (iy’, i.’ +++ tn’) if @ > %’ and % = % 
200k = bs 

If a system of ordering is prescribed for the partitions, the des- 
ignation of the p-th partition of an integer n is unique. If n denotes 
the total number of bulbs at a synapse, » the ordinal of the p-th par- 
tition of m according to some prescribed system, then and » deter- 
mine the synaptic type of the neuron associated with the synapse. If 
h is the threshold of the neuron, then the three integers 7, p,, h , com- 
pletely characterize the neuron. (See Figure 1.) 


aa 


FIGURE 1 


It remains to define the parameters which will characterize our 
neural net. a ; 
Probability of synaptic types. The probability that a synapse in 
an infinitesimal region about the point Q of the region containing the 
\ 
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net is of a synaptic type (n, p) will be designated by S(n,p,Q). 

Probability of thresholds. The probability that a neuron in an 
infinitesimal region about Q has threshold h will be denoted by 
T(h, Q). 

Frequency distribution of origins. The probability that a bulb 
group at Q originates in the infinitesimal region about a point X will 
be denoted by O(Q, X)dxX. 

All three parameters are functions of position Q and other vari- 
ables. It is to be noted that S and T are actual probabilities. The argu- 
ment of S besides that of position is a pair of integers; that of T a sin- 
gle integer. The parameter O , however, is a probability frequency dis- 
tribution. Thus the actual probability that a bulb group at Q origi- 
nates in a region RF’ is given by f2(@, X)dX. If we suppose that 


our net is closed, that is every bulb of the net has its origin within 
the net, we must have f,0(@ , X)dX — 1 for all functions O(Q, X), 


where the integral is taken over the entire region containing the net. 

Let now an initial state of a neural net be given at the time ¢ = 0 
in the following manner. The function f,(X) will specify the frac- 
tion of neurons in the infinitesimal region dX about X which are fir- 
ing at the time t = 0. The state of the net will be determined for any 
time ¢ if an expression for f;(X) is determined. We now seek f,(X). 

Consider a point Q in our region. The probability of firing at {= 1 
for each neuron in the infinitesimal region about Q depends on the 
distribution of synaptic types and thresholds around Q and on the 
frequency distribution of origins at Q. In fact, a neuron at Q will 
fire at ¢ = 1 if a sufficient number of bulbs terminating on it origi- 
nate at neurons firing at t = 0. The probability of a neuron at Q 
having synaptic type (n, p) is S(n, p, Q), and the probability that 
its threshold is h is T7(h). Therefore, if the two probabilities are in- 
dependent (as they are supposed to be at this time), the probability 
of a neuron at Q being determined by the integers n, p, h is 
S(n,p, Q)T(h, Q). 

Now consider the probability that a bulb group of a neuron at 
Q originates at a neuron firing att =0. This probability is given by 


1,(Q) = J, O(Q,X) fo(X) aX, (1) 


where the integral is taken over the entire region containing the net. 

Consider those neurons for which p = 1 in a lexicographic order- 
ing, that is those which have only one bulb group terminating on 
them. The probability of occurrence of such neurons and of their 
firing at t= 1 will equal S(n, 1, Q)T(h, Q)I, where h Sn. But in 
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general, where various combinations of distinct active bulb groups 
determine the firing of the neuron, the integral J will be replaced by 
a certain function of I. Let us see what sort of functions come into 
consideration. 

To fix ideas, suppose a neuron at Q has four bulbs terminating 
on it and threshold 3. Then, if all four bulbs belong to one group, 
the probability of the neuron firing at t = 1 equals the probability 
that a neuron firing at t = 0 sends an axone to the synapse at Q. This 
probability is J,. If three of the bulbs are terminals of one axone, 
the probability is still J,, since it would be the same as if the syn- 
apse contained only three bulbs in one group, because the firing or not 
firing of the fourth bulb is of no consequence. If, however, the four 
bulbs are paired off to two axones, both must originate at a neuron fir- 
ing at =0,and this probability is J,?. If the four bulbs are partitioned 
into two singles and a couple, we must have the couple and at least 
one of the singles originating at a firing neuron. This probability is 
I, (2I, — Io?) = 2I,? — I,3. Finally, if all four bulbs are singles, the 
probability of the neuron at Q firing is the probability that at least 
three of its four bulbs originate at firing neurons, which is 3/,° — 21‘ . 

Thus we see that the probability of Q firing depends on certain 
polynomials of J. These polynomials depend on n, p, and h and will 
be designated by P,,.,(J). Each set of integers (n, p, h) completely 
determines a polynomial, but different sets may determine the same 
polynomial. 

Neurons associated with the same P(I) will be said to have the 
same connectivity type. Neurons of the same synaptic type are not 
necessarily of the same connectivity type and vice versa. Neurons 
having the same (”, p, h) will be said to be of the same type, and 
are, of course, of the same synaptic and the same connectivity type. 
Thus both synaptic type and threshold determine the connectivity 
type. ; 

m By way of example we exhibit all the polynomials P,,., which 
may arise if n= 4. There are five partitions of 4, and we order them 


as follows: 


p= 1 p= By p= 3 p=—4 p= 5 
(0,0,0,1) (1,0,1,0) (0,2,0,0) (2,1,0,0) (4,0,0,0) 


Then 
Puy = 1 Pal Po 2) — 2 
Pap =1 Pog = 1 Pe 2 —f 
Py3=1 P42 = 1 Py3= 2 
Py.=l Py =P? Pi4,.= TP? 
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Puy = 31 —3P +P P,5, = 41 — 677 + 42° — I* 
I ge ged ee oe EI hl Paso == 612 — OF? + 4I* 
Pus = 2P — FF P45, = 31° — 2I* 

Pug =P Yep es hd 


Since the types of neurons characterized by distinct sets 
(n, p, h) are mutually exclusive, the probability of a neuron at Q 
firing at the time t = 1 will be given by the sum of the probabilities 
of its being of each type and firing. 

Now we assume that the number of neurons in the region at Q 
is large compared with the number of bulb groups originating at the 
same neuron. This assumption enables us to say that the fraction of 
neurons firing in the region about Q at t — 1 equals the probability 
of a neuron in that region firing at t= 1. 

We can now give an analytic expression for f,(Q) in terms of 
fo(Q) and the parameters of the net: 


n=0 p=p(n) h=n 
A(Q)==f FT S(n,p,Q) ST (h, Q) Pamllo(Q) 1], (2) 
A nm=1 pl h=1 


where 


1.(Q) = f fn(X)O(Q, X) dX. 


III. 

In what follows we shall illustrate some implications of equation 
(2) by applying it to some very simple cases. 

Case 1: Constant initial firing distribution; constant threshold; 
equiprobability of n; p= 1. 

Consider a neural net with the following properties. 

1. The number of bulbs terminating on any neuron does not 
exceed N . 

2. The number of bulb groups at any synapse does not exceed 
one. 

3. All the synaptic types {n, 1}, where n = 1---N, are equally 
probable. 

4. The threshold of every neuron is h*, a constant. 


5. The initial state of the net f.(X) = f., a constant through-. 
out the region. 


By the above assumptions we have 
S(n,1,Q)=1/N; S(n,p,Q)=Oforp¥l1, (3) 
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PU Q) =o: (4) 
I= { £.0(Q,X)dX=fy. (5) 
R 
Thus equation (2) reduces to 
tex N—A*+1 
fi:(Q) = ~ (1/N) > 6, widtee By eS (6) 
Since the process is reiterative, we obtain 
vee N—h*+1 ; 7 
i= N fa (7) 
And by induction 
p=(“*)s (8) 
ae N O° 


Obviously f; will be independent of ¢ if and only if h* = 1. Other- 
wise f; will tend to zero as a geometric sequence with the ratio 
(N — h* + 1)/N. In this case the ratio is independent of f;. We 
shall give examples below where it does depend on f;. In all cases 
we shall refer to the ratio f;.:/f; as the firing ratio (in general a func- 
tion of f;). If the firing ratio is always less than unity, we shall refer 
to it as the decay factor. 

Case 2. Equiprobability of nandh. Threshold of any neuron 
does not exceed H. 

If we suppose the same net as in Case 1, except that the thresh- 
olds are randomly distributed, we obtain after similar computation 
the firing ratio for this net, which turns out to be also a decay factor 
independent of f;, namely, 


2 —H 
a jt (9) 


-foalhe=| - 


Thus 
+1—H\t 
p=(=——) fo, independent of tif H=1. 


The decay factors in Cases 1 and 2 are equal if h = Fae! kl ae 


which is intuitively evident. 
Case 3. Bernoulli Distribution of Thresholds ; ior aS of 


1? Dex — bmx = N. 
We have in this case 
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1 Wie 
a (10) 
rov=ga( So) 
Sih pie ) 
ETH =ZS ae ee E 
which gives for the decay factor 
1 n=N r=n-1 N — t 
fief =— BS —) (| ). (11) 
This can be aes as a single sum as follows: 
Tr=n—1 N neta 1 
fen/fe= Sia) | ). (12) 
N22 CG 


Case 4. Equiprobability of h; Bernoulli distribution of n. In this 
case 


T(h) =1/N, (13) 
1 N—-1 


and the decay factor is ee by 


ee on = el Nowa n—1 : (15) 


By symmetry of binomial sae it follows that 


n=N N—1 T=N-1 P wi 
> n( )= > w—n(% Ni (16) 
i Vi 


n=1 7T=0 


so that the decay factors in Cases 3 and 4 are identical. 
Case 5. Bernoulli Distribution for both n and h. 
This case yields the decay factor 


AEN = VE CN as eee Nis 1 PN eaegh 
rel Oe PA OY es | CP 


N—1 N—2 N et 
sac? ate hace) | Carma ier cr: 
1 2 i 
Case 6. Connectivity Types Determined by Two Equal or Nearly 
Equal Bulb Groups at Every Synapse. 
Consider a neural net in which every synapse contains exactly 


two bulb groups with either equally distributed bulbs (if n is even) 
or differing by one bulb (if m is odd). Let also all thresholds and 
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bulb numbers be equiprobable. The maximum number of bulbs oc- 
curring, N , will be taken as an even integer for simplicity (the ar- 
guments for an odd integer are very similar). Then all the neurons 
of the net which can fire at all are divided into the following connec- 
tivity types. 


} 


1. Where the activity of either group is sufficient to fire the neu- 
ron. Here P(J) = 2] — [2. 

2. Where the activity of the larger group is necessary and suf- 
ficient to fire the neuron. Here P(J) =1. 

3. Where the activity of both groups is necessary to fire the neu- 
ron. Here P(J) = Ff. 


To the first connectivity type belong those neurons in which 
h < 4(n + 1). To the second belong those where h = $(n + 1). 
To the third belong the neurons characterized by N2k > 4(n + 1). 
Here for a constant initial distribution we obtain ; 


He 
rT RE cap ae = 


eA S(2n) 5 LT (h) + ) “S(2n +1) 5 : (nh) | (21, — I?) 


n=1 h=1 n=1 


+["S" se@n—nr(m) |r 4) SS(2n) S T(h) (18) 


> 71 h=nt+1 
» 


n=N/2-1 
a7 = S(2n + 1) "7 (a) | 12 
h=n+1 
Since J; = f;, we obtain for the firing ratio after laborious but ele- 
mentary calculations 
4N + f;(N —2) 
rere (19) 
Since f, = 1, the firing ratio is always less than unity and again 
lim f; = 0. 


to 


IV; 

The presence of the decay factor in Cases 1-6 is due to the fact 
that our neural nets contained neurons which can never fire, i.e., 
those whose thresholds exceed their bulb numbers. The proportion 
of such neurons is determined by the distributions of synaptic types 
and thresholds. It turned out in the cases considered that the pro- 
portion of “non-functional” neurons was so high that decay factors 
were always present. We are interested in determining conditions 
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under which there would be no decay factor, i.e., we wish the firing 
ratio fiu/f, 2 1. To simplify the problem we shall prescribe not 
distributions of synaptic types and thresholds but a distribution of 
connectivity types. It will be shown in the case under consideration 
that this distribution determines the firing ratios. 

We wish every neuron to be characterized by a connectivity type. 
Therefore, we will assign to non-functional neurons the connectivity 
type 0 (iie., the polynomials P(I) associated with them are identi- 
cally zero). If Any represents the fraction of neurons in the net of 


type (n;p,h) wehave > dnp, —=1. Furthermore, we can write equa- 
. npn 
tion (2) in a simpler form, namely 


f.= DS anwPra Uo) , (20) 
n,p,h 
where some P,,,;, may be zero. 

In the cases considered above the quantities An», were not pre- 
scribed in advance but were computed from the probability distribu- 
tions of synaptic types and thresholds. We wish now to consider the 
relation between the distribution of connectivity types and the overall 
behavior of the net. For this reason, instead of prescribing the dis- 
tributions of synaptic types and thresholds, we shall prescribe the 
distribution of connectivity types for our neural net. 

Case 7. Prescribed Distribution of Connectivity Types. 

Let 2), 41, As, As be the respective fractions of neurons having 
connectivities 0, J, J? and 2J — J? respectively. Let there be no other 
connectivity types and suppose as previously a constant initial firing 
distribution f,. We have obviously 

i=8 
A,=1. (21) 
41=0 
Let us put4, +4, +4, = 4. 
Since I, = f, , we have 


fi =Arfo + Ache? Aah olo — Fads 


22 
= fo (Ar + 2As) ++ fo? (Aas) ¢ ou 
and in general 
fie =f (a + 2s) ‘+ f7? (A — As). (23) 
The firing ratio is given by 
fie/f, A, + 2d + fi(A2—As). (24) 


Let 4; > 4,. Then the firing ratio will exceed unity if 
A, + 243 > 1— fi (Ae — As) =14 f(A; —A); (25) 
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: ae (26) 
Substituting the value of 4; from equation (21) into (26), we obtain 
as a condition for a firing ratio greater than unity 


2A — dy — do —1 1-4 


has aS Serene 
A—2h,— At A— 2). — da 


(27) 
Note that since 4; > 42,4 > 24. + 4,. Alsol ZA. 


Hence Sada oO ele now led. A — 2A, A, condition (27) 
A ae Zhe wee at 
is possible to satisfy. This implies that 1 < 24; + 4,, which is a 
stronger condition than A; >J,. 
A similar argument shows that the firing ratio will be less than 
unity provided 4; > 4, and 1 > 24, + 4,. Suppose now d’2 > A,;. Then 
in equality (25) reduces to 


1—24;—d, 1-21 +2, 4 2d, 


= ; 28 
fe ies: Oi jee) Shy 
f;>1i+ aes eee (29) 

ie Ey St mia : 


which is impossible. We have thus deduced the conditions on f; and 
on the distributions of connectivity types for the case considered such 
that the firing ratio is greater or less than unity. They are sum- 
marized as follows: 

A necessary and sufficient condition under Case 7 for the firing 
ratio to exceed unity is 


lL) 
ee Rare 


In particular if 4 = 1, 4; > 4, we have Lim f; = 1, that is eventually 


tow 


every neuron in the net will be firing. 


Lie lds da Ts (30) 


V. 


The examples considered here were especially simple because of 
the triviality of the initial firing distribution (a constant throughout 
the region) and of the synaptic types and thresholds (likewise con- 
stant throughout the region). This situation yields the simple rela- 
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tion I; = f;, which allows the calculation of successive states by, sim- 
ple iteration. If, however, f)(X) is a non-trivial function of position, 
or the neuron types are distributed non-uniformly, the problem is con- 
siderably complicated. The general expression for f,(Q) is given by 
equation (2) or equation (20). In the latter equation the coefficients of 
Pings are certain quantities Anpn(Q) which represent the fractions of the 
totality of neurons which are characterized by the connectivity type 
polynomials P,y,>,[Z(Q)]. These fractions can be in principle computed 
from the distributions of the synaptic types and thresholds. If they 
are assumed as known, equation (2) takes the form 


f.(Q) = ¥ dan @)Pan | (0) | (31) 
n,p,h 
where in general 7,(Q) is a function of Q. 

A question of fundamental interest is the possibility of existence 
of a steady state of neural activity, given the parameters of the net 
and an initial firing distribution. 

Such a steady state, constant with respect to time( if it exists) 
will be given by the solution of the integral equation 


f(Q)= 5 Japs (Q) Pra | J f(X)0(Q, X)aX | (32) 
n,D,h R 


since the equation implies that f,.4.#—= f;. 

In general the equation is non linear, since the polynomials Php, 
are in general of degree greater than one. 

The equation reduces to a linear one if all connectivities are of 
the type Pn» (IZ) =I, which implies that the firing of each neuron is 
determined by the activity of a single bulb group. 

The solution of equation (32) establishes only the possibility of 
a steady state. Questions of convergence toward such a state from a 
given initial distribution must be separately considered. 

The authors are indebted to Dr. L. J. Savage for a critical dis- 
cussion of this paper. 

This work was aided by a grant from the Dr. Wallace C. and 
Clara A. Abbott Memorial Fund of The University of Chicago. 
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